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Abstract 

The teaching of research methods is common across multiple fields in the social and edu¬ 
cational sciences for establishing evidence-based practices and furthering the knowledge 
base through scholarship. Yet, specific to mixed methods, scant information exists as to 
how to approach teaching complex concepts for meaningful learning experiences. Thus, 
the purpose of this mixed research study was to examine strategies used by selected U.S.- 
based leading mixed methodologists in mixed research courses as related to significant 
learning goals for course design. In addition, we examined the extent that their philosoph¬ 
ical stances influenced teaching and course objectives for student learning. Participants 
were 12 leading mixed methodologists who were instructors of mixed research courses 
from various institutions representing multiple conceptual stances applicable to the teach¬ 
ing of mixed research. Instructional practices are presented for instructors to consider 
when designing and teaching mixed research and general research methodology courses 
across disciplines. 

Keywords: Mixed methods research, mixed research, pedagogy, learning goals, concep¬ 
tual stance, philosophical assumptions. 


The last decade has witnessed an exponential increase in the number of mixed methods 
research studies - hereafter referred to as mixed research - in the published literature (i.e., 
journal articles, book chapters, books). Indeed, Ivankova and Kawamura (2010), who ex¬ 
amined five major databases (i.e., PubMed, ERIC, Psychlnfo, Academic One File, and 
Academic Search Premier) representing 10 subject areas (i.e., business, communication 
studies, education, health and medicine, library studies, political studies, psychology, so¬ 
cial work, sociology, and women’s studies) and two mixed research journals (i.e., Jour¬ 
nal of Mixed Methods Research and International Journal of Multiple Research Ap¬ 
proaches) from January 2000 to April 2009 documented that the number of methodologi¬ 
cal mixed research articles increased from 3 in 2000 to 26 in 2006 and 22 in 2008. In ad- 
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dition, the number of empirical mixed research articles increased from 10 in 2000 to 243 
in 2008. 

Despite this increase in the number of published mixed research works, there are still 
some topics related to mixed research that have received relatively little attention. In par¬ 
ticular, Teddlie and Tashakkori (2010) identified the following nine important issues or 
controversies in contemporary mixed research: (a) conceptual stances; (b) the conceptu¬ 
al/methodological/methods interface; (c) the research question or research problem; (d) 
language; (e) design issues; (f) analysis issues; (g) issues in drawing inferences; (h) prac¬ 
tical issues in the application of mixed research (e.g., pedagogy, collaboration, other 
models, funding); and (i) cross-disciplinary and cross-cultural applications. However, of 
these nine issues/controversies, we believe that the most important issue pertains to peda¬ 
gogy because the ways in which students of mixed research are trained will play an im¬ 
portant role in shaping the future of researchers across multiple disciplines in the social 
and educational sciences. 

Thus, it is surprising that relatively few works have been published in the area of peda¬ 
gogy in mixed research even though research, per se, influences best practices and con¬ 
tributes to essentially every field. Interestingly, slightly more than 10 years ago, in their 
thought-provoking chapter that appeared in the seminal first edition of the Handbook of 
Mixed Methods Research (Tashakkori & Teddlie, 2003a), Creswell, Tashakkori, Jensen, 
and Shapley (2003) observed that pedagogy of mixed research and research pertaining to 
the nature of mixed research courses need greater attention, and initiated a call for further 
research in this area. However, in the 6 years (i.e., 2004-2009) that followed Creswell et 
al.’s (2003) call, only five works had been published in the area of mixed research peda¬ 
gogy. Encouragingly, in 2010, James H. Davidson, Publishing Director of the Interna¬ 
tional Journal of Multiple Research Approaches, provided some leadership in this area 
by publishing a special issue on teaching mixed research that was guest-edited by Leech, 
Onwuegbuzie, Hansson, and Robinson (2010). This special issue contained nine articles 
that more than doubled the number of pedagogical works. However, despite this surge in 
works in 2010, the number of published works in this area remains relatively scant. As 
conveyed by Earley (2007), instructors of research methodology courses need a better 
understanding of how to teach students to address research from more than a mono¬ 
method design. 

Building on the works of Onwuegbuzie, Frels, Leech, and Collins (2011), we conducted a 
comprehensive review of literature databases representing numerous social and behavior¬ 
al science disciplines (e.g., business, education, psychology, social work, sociology, 
health and medicine, political studies, library studies, communication studies) for the 
years for which records existed, which revealed only 20 works, to date, devoted predomi¬ 
nantly or exclusively to the topic of teaching mixed research-based courses. These 20 
works are presented in Table 1. Further, Figure 1 displays the frequency of pedagogical 
works published per year and the focus of each work. These 20 pedagogical works repre¬ 
sent a very small percentage (i.e., approximately 2%) of the 802 mixed research articles 
(113 methodological and 689 empirical) published between 2000 and 2009 that were 
identified by Ivankova and Kawamura (2010). 
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Table 1. Citation Map of Works Published Pertaining to Pedagogy of Mixed Re¬ 
search. 



Citation 

Type of Work 

1 . 

Bazeley (2003) 

Conceptual/Theoretical 

2. 

Tashakkori & Teddlie (2003b) 

Conceptual/Theoretical 

3. 

Creswell, Tashakkori, Jensen, & 
Shapley (2003) 

Multiple mixed research-based courses or 
workshops at several institutions 

4. 

Onwuegbuzie & Leech (2005) 

Conceptual/theoretical 

5. 

Earley (2007) 

Single mixed research-based course or 
training program 

6. 

Niglas (2007) 

Multiple mixed research-based courses 
taught at the same institution 

7. 

Christ (2009) 

Multiple mixed research-based 
courses taught at the same institution 

8. 

Collins (2010) 

Conceptual/Theoretical 

9. 

Greene (2010) 

Conceptual/Theoretical 

10. 

Leech & Onwuegbuzie (2010) 

Conceptual/Theoretical 

11. 

Mertens (2010) 

Conceptual/Theoretical 

12. 

Onwuegbuzie, Leech, Murtonen, & 
Tahtinen (2010) 

Conceptual/Theoretical 

13. 

Leech, Onwuegbuzie, Hans son, & 
Robinson (2010) 

Conceptual/Theoretical 

14. 

Christ (2010) 

Single mixed research-based course or 
training program 

15. 

Hansson (2010) 

Single mixed research-based course or 
training program 

16. 

Ivankova(2010) 

Single mixed research-based course or 
training program 

17. 

Baran (2010) 

A series of quantitative and qualitative 
research courses taught at the same insti¬ 
tution 

18. 

Coronel Llamas & Boza (2010) 

Research methods in general 

19. 

Onwuegbuzie, Lrels, Leech, & Col¬ 
lins (2011) 

Multiple mixed research-based courses or 
workshops at several institutions 

20. 

Lrels, Onwuegbuzie, Leech, & Col¬ 
lins (2012) 

Multiple mixed research-based courses or 
workshops at several institutions 


Most importantly, although each of these 20 works provides extremely useful information 
that adds to the knowledge base in the area of mixed research pedagogy, as noted by 
Onwuegbuzie et al. (2011) and as seen in Table 1, with two exceptions (i.e., Creswell et 
al., 2003; Onwuegbuzie et al., 2011), all of these works are either conceptual/theoretical 
in nature, or they describe a single mixed research-based course or training program, 
multiple mixed research-based courses taught at the same institution, or a series of quan- 
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titative and qualitative research courses taught at the same institution. However, none of 
the empirical works involved the comparison of pedagogy from multiple mixed research 
courses taught at various institutions. Thus, the transferability of the findings from these 
works - each of which could be described as essentially representing an intrinsic case 
study (Stake, 2005) - could not be assessed directly by their authors. Creswell et al.’s 
(2003) research in the area of mixed methods pedagogy offered insight for future re¬ 
search and discussion pertaining to how best to teach optimally the “new and more com¬ 
plicated designs” (p. 633) to students. 


2003 |—| 

2004 

2005 

2006 

2007 

2008 

2009 

2010 
2011 

0 1 2 3 4 5 6 7 8 9 10 1112 



Total Per Year 

■ General Research 

■ Series of Mixed 

■ Multiple Mixed 

■ Single Mixed 

■ Cone eptnal Theoretic al 


Figure 1. Frequency of works published per year and focus of the work. 


Onwuegbuzie et al. (2011) responded to Creswell et al.’s (2003) call for more infor¬ 
mation regarding the content and nature of mixed research courses. These researchers 
utilized mixed research techniques to compare and to contrast pedagogical approaches 
used by instructors in mixed research courses as well as documenting the learning experi¬ 
ences of students enrolled in a mixed research course. Among the numerous findings 
emerging from the instructor interview data was the emergence of a three-dimensional 
model for categorizing and organizing pedagogical approaches used in mixed research 
courses (cf. Figure 2). These three dimensions represented the following three 
metathemes - each representing a continuum - that emerged from the constant compari¬ 
son analysis: Orientation (i.e., the extent to which the instructor instills the importance of 
understanding qualitative and quantitative research traditions before mixing research ap¬ 
proaches), Level of Application (i.e., the degree to which the mixed research course was 
taught in an applied manner), and Level of Structure (i.e., the degree to which the mixed 
research course was structured). 
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Methodology Focused 
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Dimension #1 

Orientation 


Research 

Question Topic Focused 



Conceptual 


Figure 2. A three-dimensional model for categorizing and organizing pedagogical 
approaches used in mixed research courses. Directionality of the continua across each 
dimension is arbitrary. There is no intentionality of suggesting superiority of one contin¬ 
uum point or extreme over another. Rather, the appropriateness of the continuum point 
depends on the mixed research instructor. Encircled numbers represent eight possible 
combinations of the extreme joints on the three dimensions of orientation, level of struc¬ 
ture, and level of application." 


According to Onwuegbuzie et al. (2011), the number of prior research methodology 
courses taken by students directly impacts the quality of mixed methodological disserta¬ 
tion research proposals. Further, the vast majority of students in this study (91.7%) re¬ 
ported positive course experiences and expressed positive perceptions about mixed re¬ 
search. Onwuegbuzie et al.’s (2011) mixed research study demonstrated the utility of 
conducting instrumental case studies - specifically, multiple/selective case studies - to 
study pedagogical approaches in mixed research courses. Building on Onwuegbuzie et 
al.’s (2011) inquiry, the purpose of the present study was to reveal and to examine the 
concepts and associated pedagogical strategies that are deemed important to teach from 
the perspectives of leaders in the field of mixed research. Through this inquiry, it was our 


2 This figure was adapted from Onwuegbuzie, Frets, Leech, and Collins (2011). 
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hope that instructors of mixed research courses - as well as instructors of comprehensive 
research courses - develop a greater understanding of philosophical/conceptual perspec¬ 
tives and pedagogical strategies that address significant learning experiences. 

Research Questions 

Qualitative research questions. The following qualitative research questions were ad¬ 
dressed in this study: 

1. What are the dominant learning goals in mixed research courses used by selected 
U.S.-based leading mixed methodologists? 

2. What are similarities and differences in pedagogical strategies used by selected 
U.S.-based leading mixed methodologists? 

Mixed research question. The following mixed research question was addressed in this 
study: 

What are the dominant learning goals in mixed research courses used by selected 
U.S.-based leading mixed methodologists as a function of conceptual stance? 

Conceptual Framework 

Our study was framed using Fink’s (2003) six categories of learning goals as outlined by 
Earley (2007) in his seminal article for developing a syllabus. Fink (2003) developed a 
model of integrated course design that involves a 12-step process for creating significant 
learning experiences that are characterized by the following three key sets of recursive 
decisions: learning goals, teaching and learning activities, and feedback and assessment. 
The second of the 12 steps - which is part of the initial phase (i.e., building strong prima¬ 
ry components) in integrated course design - represents a particularly important step. This 
step involves formulating significant learning goals. According to Fink (2003), instead of 
identifying a list of topics that students should learn about or master, course instructors 
should use a “learning-centered approach and identify what students should get out of the 
course” (p. 73). As such, Fink (2003) advanced a taxonomy of significant learning that 
promotes the following six types of significant learning: (a ) foundational knowledge (i.e., 
“knowledge about the phenomena associated with the subject and the conceptual ideas 
associated with those phenomena” [p. 74]); (b) application (i.e., “an ability to use and 
think about the new knowledge in multiple ways, as well as the opportunity to develop 
important skills” [p. 74]); (c) integration (i.e., “the ability to connect one body of 
knowledge with other ideas and bodies of knowledge” [p. 74]); (d) human dimension 
(i.e., “discovering how to interact more effectively with oneself and with others” [p. 74]); 
(e) caring (i.e., “the development of new interests, feelings, and values” [p. 74]); and (f) 
learning how to learn (i.e., developing the knowledge, skills, and strategies for continu¬ 
ing one’s learning after the course is over” [p. 74]). Fink’s taxonomy was used in the cur¬ 
rent study to identify the kinds of teaching and learning strategies and the feedback and 
assessment activities that each instructor who participated in the study utilized to support 
significant learning in her or his mixed research course. 
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The research philosophical stance for our study was what Johnson (2011) labeled as dia¬ 
lectical pluralism, referring to an epistemology that requires the researcher to incorporate 
multiple epistemological perspectives within the same inquiry. We believed that our dia¬ 
lectical research philosophical stance is compatible with Fink’s (2003) model of integrat¬ 
ed course design because this model “builds on and incorporates many ideas that already 
exist in the published literature on instructional deign and good teaching” (p. xiii). 

Theoretical Framework 

Teddlie and Tashakkori (2010) identified the following six contemporary conceptual 
stances associated with mixed research: a-paradigmatic stance, substantive theory stance, 
complementary strengths stance, multiple paradigms stance, dialectic stance, and alterna¬ 
tive paradigm stance (formerly called single paradigm stance). Each of these stances is 
summarized in Table 2. As stated by Teddlie and Tasha kk ori (2010), each of these six 
conceptual stances “has been used (explicitly or implicitly) by groups of scholars who are 
practicing MMR [mixed methods research]” (p. 14). 


Table 2. Teddlie and Tashakkori’s (2010) Six Conceptual Stances Associated with 
Mixed Research. 


Conceptual Stance 

Description 

A-paradigmatic 

Paradigms or conceptual stances are not important to 
read-world practice 

Substantive theory 

Theoretical orientations (e.g., critical race theory) are 
more pertinent to the underlying research study than 
are philosophical paradigms 

Complementary strengths 

Mixed research is possible but different approaches 
must be kept as separate as possible in order for the 
strength of each paradigm to come to the fore 

Multiple paradigms 

A single paradigm is not appropriate for all mixed 
research designs; rather, different paradigms are 
relevant for different mixed research designs 

Dialectic 

Use of multiple paradigms in a single mixed research 
study yields greater understanding of the underlying 
phenomenon 

Alternative paradigm 

Single paradigm (e.g., pragmatism-of-the-middle; 
transformative emancipator) is used to support the use 
of mixed research 
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Teaching per se represents an epistemology whereby the essence of knowing occurs dur¬ 
ing the act of teaching. Because there is a strong connection between philosophy and 
teaching, with epistemologies providing underlying theoretical frameworks for teaching 
(Kincheloe, Slattery, & Steinberg, 2000), it is likely that an instructor’s research philoso¬ 
phy plays an important role in the pedagogical approaches used in mixed research cours¬ 
es. Thus, it was of interest to explore this potential link for understanding better potential 
learning outcomes relevant to research methodology coursework. 


Method 

Participants and Setting 

The participants were 12 leading mixed methodologists from various institutions in the 
United States who were instructors of mixed research courses. They were selected via a 
criterion sampling scheme (Miles, Huberman, & Saldana, 2014) due to their knowledge 
and skills in mixed research, which includes integration of both qualitative and quantita¬ 
tive research methodologies. As demonstrated by Guest, Bunce, and Johnson (2006), 12 
interviews are sufficient to “understand common perceptions and experiences among a 
group of relatively homogeneous individuals” (p. 79). Also, as concluded by Johnson and 
Christensen (2010), “when greater resources are available, collective case studies of 
around 10 cases are common” (p. 397). Thus, our sample size of 12 instructors was 
deemed adequate for obtaining data saturation. 

The 12 instructors comprised six women and six men, who ranged in rank from assistant 
professor to full professor, teaching mixed research either: site-based (i.e., face-to-face), 
online, or hybrid (combination of face-to-face and online) context. Using the Carnegie 
Classification (The Carnegie Foundation for the Advancement of Teaching, n.d.), we 
noted the instructors’ affiliations as: institutions with very high research, institutions with 
high research, institutions with doctoral-level research, or institutions wherein research is 
not classified. With respect to the mixed research course syllabi, sample learning out¬ 
comes based on course objectives included that students would successfully: 

• Understand the historical underpinnings of mixed research. 

• Compare and contrast mixed research to mono-method research. 

• Describe the major steps in the mixed research process. 

• Evaluate several ways of collecting data in mixed research studies. 

• Demonstrate ways to address legitimation in mixed research. 

• Identify ethical and legal considerations involved in conducting and reporting re¬ 
search. 

Using Teddlie and Tashakkori’s (2010) typology, with respect to mixed research concep¬ 
tual stances, the composition of participants was as follows: five participants could be 
classified as endorsing a dialectic stance, four participants could be classified as support¬ 
ing the alternative paradigm stance, two participants could be classified as promoting the 
multiple paradigms stance, and one participant could be classified as advancing the com¬ 
plementary strengths stance. Thus, four of Teddlie and Tashakkori’s (2010) six 
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Table 3. The 12 Participants and Associated Conceptual Stance. 


Conceptual Stance 

Number of 
Participants 

Identified in Stance 

Identifying Name for 

Each Participant 

Alternative paradigm 

4 

Participant API 

Participant AP2 
Participant AP3 

Participant AP4 

Dialectic 

5 

Participant D1 

Participant D2 

Participant D3 

Participant D4 

Participant D5 

Complementary strengths 

1 

Participant CS1 

Multiple paradigms 

2 

Participant MP1 
Participant MP2 

A-paradigmatic 

0 


Substantive theory 

0 



conceptual stances were represented by the 12 participants. Table 3 presents the partici¬ 
pants and each participant’s associated conceptual stance. 

Instruments and Procedures 

The 12 leading mixed methodologists were interviewed via one of three modes: face-to- 
face, telephone, or Internet virtual meeting. Regardless of mode, these interviews were au¬ 
dio-taped using two separate hand-held digital recorders to ensure clarity of recordings. 
The interviews were semi-structured in nature and consisted of open-ended questions. In 
addition to asking these questions, the interviewer(s) probed each participant’s responses in 
order to obtain richer data and to facilitate both theory-generation and theory-confirmation. 
Participants provided information about learning outcomes via syllabi and interview re¬ 
sponses. Approval to conduct the interviews was obtained from the Institutional Review 
Board at the institutions of two of the researchers who conducted this study. Samples of 
interview questions are as follows: 

1. What types of pedagogical techniques do you believe facilitate students’ learning 
about mixed methods? How effective do you believe they are? 

2. What are the issues for instructors in designing and delivering courses that aim to 
develop researchers’ abilities to carry out mixed methods? 
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The transcribed interviews underwent member checking (Lincoln & Guba, 1985) in order 
to maximize descriptive validity (Maxwell, 1992). Further, recognizing that the research¬ 
ers were the primary research instruments (Poggenpoel & Myburgh, 2003), the two re¬ 
searchers who were involved in interviewing the participants underwent debriefing inter¬ 
views themselves, as conceptualized by Onwuegbuzie, Leech, and Collins (2008). Ac¬ 
cording to Onwuegbuzie et al. (2008), debriefing interviews are designed to promote re- 
flexivity; to identify biases in interpretation of data; and to obtain rich insights as to ways 
that the study impacted participants, stakeholders (i.e., instructors and students of mixed 
research courses), and the researchers themselves. 

Mixed Research Design 

The present study utilized a qualitative-dominant mixed research design (Johnson, 
Onwuegbuzie, & Turner, 2007). As conceptualized by Johnson et al. (2007), in this 
study, the researchers adopted a qualitative, constructivist-poststructuralist-critical stance 
with respect to the research process, while, at the same time, considering the addition of 
quantitative research approaches in general and quantitative analyses in particular to yield 
value-added inferences. 

Mixed Data Analysis 

Being dialectic pluralists, the researchers utilized mixed analysis techniques - specifical¬ 
ly, a sequential mixed analysis (Onwuegbuzie & Combs, 2010) - to address the research 
questions. Specifically, the qualitative analyses preceded the quantitative analyses. 

Qualitative phase. A series of classical content analyses (Berelson, 1952) was used to 
extract the themes pertaining to each qualitative research question. The sources for nam¬ 
ing codes and locus of typology (i.e., theme) development were literature (i.e., based on 
Fink [2003]) and investigative (i.e., stemming from the intellectual constructions of the 
researchers [Constas, 1992]) via an iterative process involving both a priori and a posteri¬ 
ori coding. Also, the verification component was referential (i.e., based on Fink [2003]) 
and technical (e.g., use of intercoder agreement). Specifically, two of the researchers in¬ 
dependently coded 20% of the interview data and after establishing 100% interrater relia¬ 
bility, using Cohen’s Kappa measure (Siegel & Castellan, 1988), one researcher coded 
the remaining interview data. Further, all the participants’ data were subjected to cross¬ 
case analyses (Miles et al., 2014) to determine whether the emergent themes could be 
disaggregated. The QDA Miner 3.2 software program (Provalis Research, 2009) was 
used to analyze the qualitative data. 

Mixed analysis phase. In the mixed analysis phase, the codes extracted from the qualita¬ 
tive analyses were analyzed quantitatively. In particular, the codes extracted from the in¬ 
terviews were subjected to a correspondence analysis, which is a technique for conduct¬ 
ing a mixed analysis of emergent themes (cf. Michailidis, 2007). We utilized the QDA 
Miner 3.2 software program (Provalis Research, 2009) to conduct the correspondence 
analysis. Thus, our analysis represented a form of crossover mixed analysis (Onwuegbuz¬ 
ie & Combs, 2010) because data associated with one tradition (i.e., qualitative, content 
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analysis) were analyzed via another tradition (i.e., quantitative analysis, correspondence 
analysis). 


Results 

In creating his taxonomy of significant learning, Fink (2003) emphasized that learning is 
defined in terms of change for students. As such, each of his six major areas of change 
contains between two and five descriptive subgoals/subcategories that yield a total of 18 
subgoals. It is important to note that in his taxonomy, the significant learning goals are 
not hierarchical but are relational and interactive. Therefore, often times, teaching strate¬ 
gies might be considered in one or more categories. The following sections are organized 
by research question and learning goals/subgoals as they relate to teaching strategies dis¬ 
tinguished through our findings. 

Qualitative Research Question 1: What are the dominant learning goals in mixed re¬ 
search courses used by selected U.S.-based leading mixed methodologists? 

Application. Fink (2003) defined Application, which comprises five specific subgoals 
(i.e., Critical Thinking, Important Skills, Practical Thinking, Creative Thinking, and 
Complex Projects). He established that Application is the second most common educa¬ 
tional goal for many college instructors - second to foundational knowledge. In addition, 
he emphasized that Application extends beyond the idea that students are learning to use 
knowledge, but also incorporates the idea that application, or using foundational 
knowledge, includes developing: Skills, Critical Thinking, Creative Thinking, Practical 
Thinking, and Complex Projects. For the participants in our study, Application emerged 
as the dominant learning goal. Within the area of application, the subtheme of Critical 
Thinking was mentioned by all of the 12 participants as being integral to teaching mixed 
research. The subtheme of Practical Thinking was noted by 11 of the 12 participants, 
closely followed by Creative Thinking (10 of 12 participants) and Complex Projects (8 of 
11 participants). Regarding the most frequently occurring subgoal, Critical Thinking, one 
participant revealed, 

And my goal is to get students to conceptually understand what a factor analysis 
is on one hand and a content analysis on the other hand, and how really you're do¬ 
ing the same thing. I think that the conceptual understanding of these similarities 
and their differences in methodological orientations is probably the most im¬ 
portant thing. (Participant API) 

The learning subgoal of Creative Thinking often included a strategy to address a stu¬ 
dent’s leaning toward a specific research philosophy: 

So, they start off with very often broad and expansive or an impossible design 
which introduces the concept of the parameters of research designs and bounding 
the case. Their conceptual stance is what they bring into the class before they've 
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done the readings, often preferring a specific strand which sometimes limits their 
way of thinking about how to conduct research. (Participant AP2) 

In the subtheme of practical thinking, one participant specified, 

In a mixed research study that is evolving, and qualitative research also has that 
dilemma, there are going to be questions that evolve as the study evolves; and as a 
researcher, you have to make those quick decisions as to ‘what should I ask; what 
is appropriate to ask; what is inappropriate to ask; what to include; what not to in¬ 
clude,’ and so forth. (Dl) 

Integration. Fink (2003) discriminated three realms of integration: (a) Within-Course 
Integration, or interdisciplinary learning (i.e., students look at problems from the perspec¬ 
tives of two or more disciplines or perspectives within a course); (b) Between-Courses 
Integration (i.e., students integrate different perspectives and focus on connecting diverse 
people as well as diverse fields); and (c) Other Areas of Life Integration (i.e., social or 
work connections stemming from academic understanding). The learning goal Integration 
emerged as the second most dominant goal emphasized by all 12 participants. In addition, 
all three subthemes (Within-Course, Between-Courses, Other Areas of Life) were empha¬ 
sized by participants within the theme of Integration. With respect to Within-Course In¬ 
tegration, one participant discerned, 

[Students] just have to learn not only the content but also they have to learn the 
language, the terminology, and actually, the concepts you know. Content is more 
methodological - how you do things differently in mixed methods research, quan¬ 
titatively or qualitatively - concepts are new things that are unique to mixed 
methods research, for example, like how you integrate. (Participant MP1) 

To address the importance of Between-Course Integration, another participant explained, 

Also, I think that if you're going to have them do a proposal or do a project, [stu¬ 
dents] have to come to the class with some sort of a content area of interest. And 
so, you are simultaneously trying to teach the students how to do mixed methods 
research and also to help them develop an area that they're interested in. So, if 
they don't come to the course with some area that they're interested in, then it be¬ 
comes sort of a hypothetical exercise. (Participant API) 

Regarding the theme of Other Areas of Life Integration, philosophy of research appeared 
to help bridge concepts of mixed research embedded within either the community at-large 
or within the personal perspective of the individual. For example, one participant sug¬ 
gested, 


I think that the core - if there is a core idea of mixed research - is listening to 
multiple perspectives and believing that something greater will be gained from 
that. Something better will be gained by listening and respecting multiple per¬ 
spectives. (Participant D2) 
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Further, another participant explained, 

I would talk about the integration of thinking that emerges from a quantitative 
perspective, and a qualitative perspective. But I would also put that within a 
framework, a larger framework. What are the underlying philosophical beliefs 
that guide you in your understandings about the meaning of mixed methods? 
(Participant AP4) 

Table 4 presents the emergent six learning goals and 18 subcategories/subthemes in rank 
order from highest in frequency to lowest in frequency relating to the participants. As 
seen in Table 4, even though the learning goal of Application was a dominant theme, the 
subtheme of Within-Course-Integration was uncovered as the most frequently occurring 
overall subtheme. When comparing the percentage of overall frequency of the 18 signifi¬ 
cant learning subgoals, it should be noted that: (a) Within-Course Integration, (b) Per¬ 
spectives for Understanding, and (c) Key Information were only slightly higher than were 
the subthemes of (a) Critical Thinking, (b) Other Areas of Life, and (c) Interacting with 
Others. 

Table 4. Significant Learning Goals and Subtotals in Rank Order of Emphasis by 
Leaders in Mixed Research Courses. 


Significant Learning Goal and Related 
Subgoals 

Rank 
of Goal 

Coding Fre¬ 
quency for 
Goal 

Coding 
Frequency 
for Subgoal 

% of Overall 
Subgoal Fre¬ 
quency 

Application 

1 

106 



Critical Thinking 



26 

7 

Important Skills 



25 

6 

Practical Thinking 



22 

5 

Creative Thinking 



17 

4 

Complex Projects 



16 

4 

Integration 

2 

90 



Within-Course 



45 

12 

Other Areas of Life 



27 

7 

Between-Courses 



18 

5 

Foundation 

3 

76 



Perspectives for Understanding 



39 

10 

Key Information 



37 

9 

Learning How to Learn 

4 

44 



Inquiry Within-Course 



16 

4 

Self-Directed Learning 



14 

4 

How to Be a Good Student 



14 

4 

Human Dimension 

5 

46 



Interacting with Others 



29 

7 

Learning about Self 



17 

4 

Changes in Caring 

6 

27 



Feelings about the Subject 



3 

1 

Interests about the Subject 



6 

2 

Values about the Subject 



18 

5 
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Mixed Research Question: What are the dominant learning goals in mixed research 
courses used by selected U.S.-based leading mixed methodologists as a function of con¬ 
ceptual stance? 

The correspondence analysis yielded interesting results as a function of conceptual 
stance. Figure 3 displays the correspondence analysis and the 18 subthemes relating to 
each participant as a function of conceptual stance. In the correspondence analysis, the 18 
sub themes and 12 participants were clustered on a horizontal axis with the following 
three of the six major learning goals, specifically identified through their related sub¬ 
goals, which were positioned mostly on the far left: (a) Learning How to Learn (i.e., In¬ 
quiry Within-Course, Self-Directed Learning, How to be a Good Student), (b) Human 
Dimensions (i.., Interacting with Others, Learning about Self), and (c) Changes in Caring 
(i.e., Feelings about the Subject, Interests about the Subject, Values about the Subject). 
On the far right of the continuum of the horizontal axis, two of three other learning goals 
(and related subgoals) are positioned: Application (i.e., Critical Thinking, Practical 
Thinking, Creative Thinking) and Foundation (i.e., Perspectives, Key Information). 
However, the learning goal Integration spans across the continuum from Inquiry Within- 
Course to Other Areas of Life. On the vertical axis, there were no apparent significant 
goals that clustered; yet, a few isolated subgoals appeared in the upper quadrants (i.e., 
Quadrant 1 and Quadrant 2): (a) Learning About Self (i.e., Human Dimension), (b) Criti¬ 
cal Thinking (i.e., Application), and (c) Values about the Subject (i.e., Caring). Further, 
the following subgoals are positioned in the lower quadrants (i.e., Quadrant 3 and Quad¬ 
rant 4): (a) Creative Thinking (i.e., Application) and (b) Feelings about the Subject (i.e., 
Caring). 

In addition, in Figure 3, participants are identified as they are positioned in four quadrants 
and with respect to the 18 subgoals. Specifically, two participants are positioned in Quad¬ 
rant 1, two participants are located in Quadrant 2, four participants are positioned in 
Quadrant 3, and four participants are located in Quadrant 4. Similar to the results of the 
content analysis, the subgoals that were clustered closest to the origin (i.e., intersecting 
axes) as central themes among participants in this area were as follows: (a) Key Infor¬ 
mation, (b) Perspectives, (c) Important Skills, and (d) Complex Projects. 

Upon closer examination of Figure 3, it can be seen that two of the three learning goals 
(i.e., Application [Critical Thinking, Practical Thinking], and Foundation [Key Infor¬ 
mation, Perspectives]) that were positioned to the right of the vertical axis are learning 
goals that pertain to what might be considered the depth of teaching. Conversely, the 
subgoals relating to the other three learning goals (i.e., Learning How to Learn [Inquiry 
Within-Course, Self-Directed Learning, How to Be a Good Student], Human Dimension 
[Interacting with Others, Learning about Self], and Changes in Caring [Feelings about the 
Subject, Interests about the Subject, Values about the Subject]), which might be consid¬ 
ered a type of breadth in teaching, appeared only on the left of the spectrum. The learning 
goal Integration represented both depth and breadth. Depicted in Figure 3, the two partic¬ 
ipants who held the Multiple Paradigms conceptual stance (i.e., MP1 and MP2) are situ¬ 
ated, respectively, in Quadrant 2 and Quadrant 4 (i.e., depth in a course). Further, 
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Human Dimensions 


Changes in Caring 


Figure 3. The 18 subgoals related to six learning goals (Fink, 2003) and participants 
by conceptual stance. The six significant learning goals are presented as they relate to 
the 18 subgoals on the horizontal continuum. Also, the quadrants are numbered 1-4. 
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participants who were classified as representing an Alternative Paradigm stance (i.e., 
API, AP2, AP3, AP4) are scattered farthest away from each other on the correspondence 
plot, appearing in three of the four quadrants. Also, the one participant with the Comple¬ 
mentary Strengths stance (CS1) is situated farthest right in Quadrant 4 and closest to one 
of the AP participants, specifically Participant AP3. Interestingly, both of these partici¬ 
pants viewed the integration of course content quite uniquely. Participant AP3 empha¬ 
sized in both his/her interview and member checking (by inserting an underline under the 
word research) the importance of presenting a mixed methods course as a research 
course, by stating the following: 

I don't call it a mixed methods course, or mixed methods research. It is research. 
It is the way it's supposed to be, rather than be fragmented. Everything is together. 
Quant is integrated in many respects. When it comes to design for example, for 
experimental design, we have both qualitative kind of data, quantitative kinds of 
examples, questions, etc. All the way to ethnography, it's the same as we go 
through designs. It's the same as much as possible-they're integrated. 

Also different from other participants, Participant AP2 explained how philosophy frames 
perspectives that are important to bridge personal beliefs and integration. This participant 
declared, 

Combining methodological traditions in mixed methods research now includes 
mixing methodologies within a particular conceptual stance. What I mean by that 
is, I'm seeing and encouraging students to consider conceiving of mixed methods 
as more than qualitative and quantitative research, it can be a mixture of para¬ 
digms, conceptual stances, or mixing methodologies within a particular traditional 
research stance such as in the qualitative domain where combining things like 
transformative or critical research stances with case study design is a mixture. 

Finally, Participant CS1, who held a different conceptual stance from all other partici¬ 
pants, revealed a unique perspective of teaching mixed research that was different from 
all other participants: 

I use the term mixed methods differently from some others. Mixed methods con¬ 
tain one complete method plus at least one other strategy. And if you're using two 
complete methods, I refer to that as ‘multiple methods.’ 

As can be seen in the correspondence plot, other differences with respect to the central 
significant learning goals can be distinguished within Quadrant 1: D1 and API. Both of 
these participants addressed bridging content with culture. As articulated by Participant 
Dl, 


I want to create a culture of research and a phrase I like to use as well is a ‘culture 
of open-mindedness,’ which I unfortunately don't see enough. With open- 
mindedness, meaning, I am willing to learn anything? I may not use it down the 
road, but I am willing to learn about software, and if I don't use it, great. But at 
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least if I don't use it, it is not out of ignorance; it is out of philosophy, or because 
it doesn't help me address my research question and so forth. 

It was discovered that the majority of the participants (i.e., 9 of 12) mentioned that stu¬ 
dents should have a thorough grounding in both quantitative and qualitative research 
skills before undertaking a course in mixed research. One participant reasoned, 

It's extremely important to me for them to fully understand the strong version of 
qualitative research and the strong version of quantitative research. What I mean 
by that is that it includes epistemological and ontological viewpoints, positions... 
in addition to the methods. So, I don't want anyone to argue that I didn't really 
teach them qualitative or say I didn't really teach quantitative or whatever it might 
be. So, after I've done that, now I'm really ready to teach them about mixed re¬ 
search. (Participant D2) 

Conversely, Participant D1 surmised, 

I think students need to have a thorough grounding in research, not quan not qual, 
but in research, because there are core things that cut across quantitative and qual¬ 
itative research, we tend to focus on differences, and that is always a frustration of 
mine, rather than focusing on commonalities. 

Thus, to distinguish further the similarities and differences, a partially ordered matrix 
(Miles et al., 2014) was created whereby we displayed conceptual stance, whether or not 
the participant mentioned the need to have a thorough understanding of quantitative and 
qualitative research before taking the course, and the dominant within-case participant 
learning goal(s). In addition, we summarized the overall impression of each within-case 
with a unique overall identified characteristic, or focus. Table 5 illustrates the findings 
pertaining to (a) the dominant learning goal(s), (b) view regarding whether a thorough 
grounding is needed in quantitative/qualitative research methods, and (c) overall charac¬ 
teristic. 

Qualitative Research Question 2: What are similarities and differences in pedagogical 
strategies used by selected U.S.-based leading mixed methodologists? 

Teaching strategies implemented by the participants included a variety of teaching tools 
for addressing the complexity of a mixed research course. The use of current published 
literature in the area of mixed research was a primary means to broaden students’ under¬ 
standing of mixed research as a process. In addition, the use of literature was used to ad¬ 
dress Critical Thinking. In fact, some participants utilized the reading of what they 
deemed as poorly designed studies versus what they considered as well-designed studies 
to address areas such as sampling procedures, data collection, types of data, and integra¬ 
tion. One participant (D2) described his/her use of readings: 

I would ask the student to find mixed research articles - empirical studies discuss¬ 
ing [one area of interest such as] bullying. Then, I ask him/her to critique these 
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Table 5. Participants by Philosophical Stance, Dominant Learning Goal, Knowledge 
of Quan/Qual Perspective, and Overall Characteristic. 


Participant by 
Philosophical Stance 

Dominant Learning Goal 

Knowledge 
necessity of 
Quan/Qual 

Overall Characteristic 

Dialectical 1 

Perspectives/ 

Interacting with others 

No 

Connected 

Dialectical 2 

Key information 

Yes 

Relevant 

Dialectical 3 

Key information/ 
Within-course/ 

Interacting with others 

Yes 

Systematic and 

Integrated 

Dialectical 4 

Key information/Other 
areas of life 

Yes 

Structured and Pertinent 

Dialectical 5 

Key Information/ 

Perspectives/ 

Within-course 

Yes 

Significant and 

Purposeful 

Alternative Paradigm 1 

Change in values/ 
learning about self 

Yes 

Reflective 

Understanding 

Alternative Paradigm 2 

Perspectives 

No 

Philosophical 

Alternative Paradigm 3 

Perspectives/ 

Within-course 

No 

Innovative 

Alternative Paradigm 4 

Perspectives/ 

Other areas of life 

Yes 

Cooperative and 

Collective 

Multiple Paradigms 1 

Within-course/ 

Critical thinking 

Yes 

Customized 

Multiple Paradigms 2 

Key information/ 
Within-course 

Yes 

In-Depth 

Complementary 
Strengths Stance 

Within-course/ 

Practical skills 

Yes 

Conceptual 


articles in order to get an understanding of how different researchers are interpret¬ 
ing this topic and how they're applying different aspects of the mixed research 
process in the critiqued studies. 

To address one strategy in teaching mixed research design, another participant summa¬ 
rized, 


Well, here's the key, I explain to students that once you have your question, you 
then figure out what design you're going to use. There are some good typology 
and designs .... And then that's going to inform a lot of phases of your project. 
You can now draw a visual picture of your procedures: how you're going to treat 
your data; where you're going to mix your data; how you think about validity; 
and how you think about ethical issues. So, the design sets the center of my 
thinking about a good mixed methods course. Now that's my particular orienta¬ 
tion. (Participant MP2) 
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Similar to the results of the correspondence analysis whereby participants clustered to¬ 
gether in depth and breadth, participants who noted the importance of quantitative and 
qualitative understanding before taking a mixed research course also surmised how ter¬ 
minology was core to understanding mixed research, as exemplified by the following 
statement: 

And I think some of the issues that students have is learning the terminology, ap¬ 
plying it appropriately, recognizing which concepts to use ... and how to construct 
dialog whether it be written or orally for communicating with others about their 
ideas. (Participant D3) 

A frequent concept to address in teaching mixed research was to move students into the 
creative realm of thinking about research. For example, Participant API concluded, 

I think the power of mixed methods research is that it can change the way that 
people conceptualize ‘what normal science is.’ They can bring to that the richness 
of qualitative data and how the quantitative and qualitative data are not antithet¬ 
ical to one another; in fact, they are just opposite sides of the same page ... The 
data are interchangeable. And so I think what the teacher or the professor has to 
do with teaching a mixed methods course is get people beyond the dichotomiza- 
tion to a point where they're simply interested in what the research problem is and 
how they can approach that research problem, from a variety of different ways. 

In addition, one central teaching strategy addressed by Participant D1 was to integrate 
learning via team-teaching with a doctoral student. He/she explained, 

I try where possible to get a student, a former student, if I can, to come into the 
class and either give some pointers at the beginning of the course, if that is all 
they have the time for, but ideally they team-teach the course with me so that stu¬ 
dents can see an example of how I try to encourage life-long researchers. They 
can interact with the students, and so the students can have an automatic role 
model via this student, who is still a student. 

Table 6 lists the various strategies in the voices of participants for the teaching and learn¬ 
ing of mixed research and the domain that each strategy addressed. Similar to Fink’s 
(2003) description that all learning goals are integrative and iterative, as depicted through 
the examples in Table 6, strategies in teaching mixed research address multiple learning 
goals. For example, foundational elements in mixed research such as design, analysis, 
and development of a study might be addressed through Critical Thinking, Practical 
Thinking, and Between-Courses Integration. In addition, participants addressed tech¬ 
niques for developing student skills such as writing research and integrating ethics 
throughout a mixed research course that might be considered as being part of the signifi¬ 
cant learning goals of Learning How to Learn, Human Dimension, and Foundation. 

To summarize further the findings pertaining to Qualitative Research Question 2, we cre¬ 
ated a matrix of instructional practices espoused by leading mixed methodologists. 
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Table 6. Techniques Used to Facilitate Important Skills and Domains Pertaining to 
the Applications of Learning Goals. 


Subtheme 

Domain 

Addressed 

Example 

Important 

Skills 

Design 

When they are through with their [project], they have learned to logi¬ 
cally think, plan and conduct research. They also are much more com¬ 
petent consumers of information. 

Important 

Skills 

Analysis 

In reference to the quantitative [approach], in my class, I only focus on 
descriptive analyses I tend to keep it at descriptive level only, to allow 
as many students [in the course] to be able to interact with the dataset. 

Important 

Skills 

Analysis 

The analyses are descriptive analyses. They don't do any conversion at 
this point, [they do] descriptive analyses, and then they augment it [this 
analysis] by a content analysis. 

Important 

Skills 

Ethics 

I should do more.. .as Joe Maxwell has said many times: the practice is 
so much more complicated than the theory. And, you know, the steps 
should be iterative. Not linear and it shouldn't be. They should be, back 
and forth and around and around. 

Important 

Skills 

Analysis 

I never expect them to collect a lot of data, but they have to collect 
some data ; their data sets don’t have to be huge and, most of the time 
they don't have enough power to find significance, and things like that, 
but I'm not that worried about that. I'm more worried about their going 
through the motions of what they will be doing in a study. 

Important 

Skills 

Design 

It's, for you as a teacher to use as examples. Present your own work is 
a tremendous boost, as you can describe the decisions, the compromis¬ 
es and the problems and how they were overcome. It is one thing to 
have a perfect design, another to be able to execute it in the research 
setting. 

Important 

Skills 

Writing 

They will get feedback on their ability to write in a technical way, 
which would include the words that they use, include grammar, include 
APA, and so forth, with a goal that once they have done that a couple 
of times, they have really started to get a feel for how to write up the 
results of a research study. 

Important 

Skills 

Analysis 

They'll be given a data set and I give them some research questions 
that are mixed and ask them to analyze the data and write it up in a 
way that would appear like it would in a journal. 

Important 

Skills 

Analysis 

I think one of the things that really helps, and there is a barrier associ¬ 
ated with that, in my present institution, is the use of software.. .like 

QDA Miner. 


Figures 4, 5, 6, and 7 each present a conceptually ordered matrix, specifically a content- 
analytic summary table (Miles et al., 2014) to illustrate: (a) activities for teaching mixed 
research (Figure 4); (b) content important to include in mixed research courses (Figure 
5); (c) design components to include in teaching mixed research (Figure 6); and (d) ethics 


The Journal of Effective Teaching, Vol. 14, No.2, 2014, 5-34 

© 

2014 All rights reserved. 



A Mixed Research Study of Pedagogical Approaches 


25 


specific to mixed research (Figure 7). It should be noted that just because an instructional 
practice is not identified by a participant from one conceptual stance does not necessarily 
mean that it is not used. However, this strategy was foremost on the mind(s) of those who 
stated it. 


Discussion 

Due to the fact that research methodology courses often involve some level of application 
and involve interacting with other disciplines, it is not surprising that our findings un¬ 
veiled the learning goals of application and integration as being highly emphasized by 
these instructors of mixed research. Importantly, due to the complex nature of mixed re¬ 
search (i.e., involving “mix[ing] or combin[ing] quantitative and qualitative research 
techniques, methods, approaches, concepts or language into a single study”; Johnson & 
Onwuegbuzie, 2004, p. 17), leaders in the field also emphasized the importance of critical 
thinking and understanding various perspectives almost as equally as they emphasized 
the importance of the subgoals pertaining to key information, application, and integration. 

The finding that the AP participants were located far away from each other in general 
highlights the way that overall philosophy of the instructor might impact pedagogy. The 
uniting element of this stance is the belief that mixed research should be driven by a dif¬ 
ferent paradigm from those paradigms associated with quantitative (e.g., postpositivism) 
and qualitative (e.g., social constructionism, critical theory) research traditions. There¬ 
fore, it is not surprising that these participants did not appear close to each other on our 
correspondence analysis plot. For example, the transformative emancipatory stance (cf. 
Mertens, 2010) is very different to various forms of pragmatism (cf. Biesta, 2010; John¬ 
son et al., 2007). Thus, it might not be surprising that they would fall into different quad¬ 
rants pertaining to learning goals. In fact, in Quadrant 3 of the correspondence analysis 
plot, it can be seen that the humanistic pieces of teaching, as we identified as breadth, is 
associated with the transformative emancipatory conceptual stance. Conversely, a prag¬ 
matist who is considered as representing the Alternative Paradigm stance might realisti¬ 
cally align with the depth quadrants that emphasize in-depth tools that tend to work when 
conducting research. This finding provides compelling evidence of the role that a teach¬ 
er’s research philosophy plays in the formation and utilization of pedagogical approaches 
in mixed research courses. 

Regarding the two Multiple Paradigm stance participants, our findings offer some insight 
into the relationship between this conceptual stance and significant learning goals. Ac¬ 
cording to the Multiple Paradigm stance, a single paradigm is not appropriate for all 
mixed research designs and that paradigms can be tailored to fit the research question and 
research design, In order to do this, a student would need a very in-depth understanding 
of multiple research paradigms before determining a best fit. Thus, it is not surprising that 
these participants fell within the right quadrants, or what we described as representing 
depth in the course content. 

Similarly, the participants who held beliefs associated with the Dialectic stance, with the 
exception of one participant, tended to fall near the origin of the plot and around the 
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Activities for Learning Mixed Research 

Conceptual 
Stance of 
Participant(s) 

Relevant 

Significant 

Learning Goals 

Discussion of research traditions and 
methodologies 

All 

Within-Course 

Structure thinking before implementing the 
study 

All 

Critical Thinking 

Use a model such as the 13-step (Collins, 
Onwuegbuzie, & Sutton, 2006) 

DP 

Practical Thinking 

Each week, use published readings to interpret 
each step of research 

All 

Critical Thinking 

First session, introduce or compose a definition 
of mixed method research 

Most 

Key Information 

Provide an overview of quantitative and 
qualitative research 

Most 

Key Information 

Be sure to balance: philosophical, conceptual, 
and applied components 

All 

Perspectives 

Integrate the course as much as possible from 
the beginning: historical, philosophical, 
worldview issues, sampling, and data collection 

Most 

Within-Course 

Integration 

Provide a variety of readings to address 
philosophical issues 

All 

Perspectives 

Include a reflective piece such as a reflective 
journal 

DP 

Learning about 

Self 


Figure 4. A content-analytic summary table of instructional practices of U.S.-based lead¬ 
ing methodologists for addressing class structure specific to mixed research. In the col¬ 
umn that notes the conceptual stance of participant(s), if more than one conceptual 
stance-participant noted the instructional practice, it is labeled as most. If all three of the 
conceptual stance-participants noted the instructional practice, it is labeled as all. 
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Practices for Addressing Content Specific to Mixed 
Research 

Conceptual 
Stance of 
Participant(s) 

Relevant Significant 
Learning Goals 

Emphasize design and validity issues, or as Jennifer Greene 
might say is a good mental model 

AP 

Perspectives 

Teach more than one conceptual stance 

All 

Perspectives 

Bring students to a deeper understanding of how you bring 
methods together 

All 

Practical Thinking 

Incorporate a type of street smart ability for students that 
they make clear all components of the study 

AP 

Other Areas of Life 

Get students to a point to recognize the overall value of 
mixing 

All 

Values about the 
Subject 

Incorporate conceptual understandings for analyses, such 
as what a factor analysis is versus a constant comparison 
analysis—as being parallel 

DP 

Between-Courses 

Integration 

Help students become methodologically eclectic 

AP 

Changes in Interests 

Use the Journal of Mixed Methods Research for examples 
of research by experienced researchers 

Most 

Critical Thinking 

Incorporate the Sage Handbook of Mixed Methods 

Research into readings 

All 

Critical Thinking 

Use a rubric to help guide students to evaluate readings 

DP 

Practical Thinking 

Have students draw the design of published articles 

CSS 

Creative Thinking 

If textbooks present mixed research in dichotomies, assign 
chapters such as 1 and 13 at the same time 

AP 

Within-Course 

Integration 

Remember that mixed research terms are new to students 
such as ontology, epistemology. Simplify these 

Most 

Key Information 


Figure 5. A content-analytic summary table of instructional practices of U.S.-based lead¬ 
ing methodologists for content important to include in mixed research courses. In the col¬ 
umn that notes the conceptual stance of participant(s), if more than one conceptual 
stance-participant noted the instructional practice, it is labeled as most. If all three of the 
conceptual stance-participants noted the instructional practice, it is labeled as all. 
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Practices for Addressing Design Components in 
Mixed Research 

Conceptual 
Stance of 
Participant(s) 

Relevant 
Significant 
Learning Goals 

Recognize sequence: a credible database, an audit trail that 
is evidence-based, integrate the findings 

Most 

Practical 

Thinking 

Clear confusion regarding sampling: what is a population 
versus a probabilistic sample? 

All 

Key Information 

Experience collecting data, especially qualitative to compare 
with other data collection experiences in quantitative 

All 

Within-Course 

Integration 

Incorporate legitimation criteria, such as Onwuegbuzie and 
Johnson (2006) 

DP 

Foundation 

Use diagrams throughout every stage of the research design 

DP 

Creative 

Thinking 

Begin with an overarching problem and design a 
complementary set of quantitative or qualitative questions, 
then integrate the findings at the end 

DP 

Other Areas of 

Life Integration 

Demonstrate how to follow through on research 

All 

Values about the 
Subject 

Help students decide mixing points, where they will mix the 
two approaches 

Most 

Critical Thinking 

Bring to students’ awareness that mixed research is a hot 
topic, but also requires a greater knowledge and greater time 
commitment 

AP 

Learning about 
Self 


Figure 6. A content-analytic summary table of instructional practices of U.S.-based lead¬ 
ing methodologists for design components to include in mixed research courses. In the 
column that notes the conceptual stance of participant(s), if more than one conceptual 
stance-participant noted the instructional practice, it is labeled as most. If all three of the 
conceptual stance-participants noted the instructional practice, it is labeled as all. 
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Practices for Addressing Ethics Specific to Mixed 
Research 

Conceptual 
Stance of 
Participant(s) 

Relevant 
Significant 
Learning Goals 

Incorporating how to address Institutional Review Board 
(IRB) with both quantitative and especially qualitative data 
collection: What is important to note? 

All 

Values about the 
Subject 

How ethics flow through data collection, data analysis, the 
write-up stage pertaining to the mixed design 

All 

Complex 

Projects/Critical 

Thinking 

Using the mixed research perspective for inspecting a 
position on ethics: examine what you value, examine the 
different ways to get there. It is an examination of the 
means and the end. 

Most 

Learning about 
SelfWalues 
about the Subject 

Examine the mixed research perspective that says to listen 
to different and multiple perspectives and construct the 
most ethically justified position of these viewpoints 

Most 

Self-Directed 

Learning 

Use role plays to communicate mixed methods to an IRB 
representative 

AP 

Creative 

Thinking 

Decide if consent is quantitatively driven or qualitatively 
driven 

DP 

Critical Thinking 

Incorporate the voices of participants and know that the 
researcher voice is always present 

AP 

Interests about 
the Subject 

Utilize Guba and Lincoln’s (1989) authenticity criteria to 
strive not only to do no harm, but moreover to do good 

Most 

Interacting with 
Others 

Inspect Mertens’s (2010) transformative-emancipatory 
stance to learn how to make a difference for your 
participants 

Most 

Values about the 
Subject 

Discuss responsibilities to gatekeepers, stakeholders, and 
also the people of whom you promise things: these are the 
same issues that are noted in ethnography and 
anthropology 

Most 

Interacting with 
Others 


Figure 7. A content-analytic summary table of instructional practices of U.S.-based lead¬ 
ing methodologists for ethics to include in mixed research courses. In the column that 
notes the conceptual stance of participant(s), if more than one conceptual stance- partici¬ 
pant noted the instructional practice, it is labeled as most. If all three of the conceptual 
stance-participants noted the instructional practice, it is labeled as all. 
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central uniting learning goals highlighted by Fink (2003) as being the most common fo¬ 
cus of most college instructors: foundational knowledge and application of that 
knowledge. Finally, the participant who adhered to a Complementary Strengths stance 
fell furthest within the depth quadrant, which also provides some evidence that this stance 
might be quite unique. Regardless, a strong research philosophy, which, Collins, 
Onwuegbuzie, and Johnson (2012) referred to as “philosophical clarity” (p. 854), seems 
to be interrelated to pedagogical strategies for these mixed research instructors. 

Potential limitations of our study included descriptive credibility, such as the academic 
time of year (October through November) because instructors might have been overly 
cognizant of one group of students versus prior groups of students. In addition, as leaders 
in the field of mixed methodology, participants might have experienced reactivity to the 
study and a heightened enthusiasm for characteristics of mixed research. Trustworthiness 
was addressed through member-checking and the use of original language of participants. 
In addition, the primary researchers addressed reflexivity through debriefing interviews 
as suggested by Frels and Onwuegbuzie (2012) in the data collection, data analysis, and 
data reporting stages of the study. 


Implications 

As distinguished by Fink (2003), good courses include “teachers who care - about the 
subject, their students, and about teaching and learning” (p. 28). Without doubt, as U.S.- 
based leaders in the field of mixed research, it is clear that the participants who engaged 
in our study care about the future of mixed research and successful learning by their stu¬ 
dents. As passionate leaders in the field, the 12 participants in our study, on average, had 
secured (a) 160.33 works published in the literature; (b) at least 5,625.17 citations for 
their works; and (c) a h-index of 23.08, indicating that, on average, at least 23 of then- 
works had been cited on at least 23 occasions (Hirsch, 2005). Although our study recog¬ 
nized pedagogy and concepts specific to mixed research (e.g., data collection methods, 
legitimation, ethics), it is important to note some of the additional overall developmental 
characteristics that emerged as being important to participants in our study. These areas 
revealed the importance of expanding perspectives, critical thinking skills, and other 
learning goals outlined by Fink (2003) for integrated course design. 

For addressing the research questions and to recognize the teaching strategies used by 
selected U.S.-based leading mixed methodologists, it was not our intention to generalize 
beyond our study. However, due to the fact that the concepts in general research method¬ 
ology courses are the building blocks for mixed research coursework, a naturalistic gen¬ 
eralization (Stake & Trumbull, 1982) might be considered by instructors of social and 
educational research methodology courses in multiple fields of study. Because mixed re¬ 
search methodology - by its very nature - is intended to expand possibilities for address¬ 
ing particular goals and objectives not possible through the use of mono-methods alone, 
the associated pedagogical strategies might reveal ways to approach multiple and complex 
research concepts. 
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The nature of mixed research is naturally integrative; thus, it is not surprising that teach¬ 
ing strategies used by mixed research instructors align strongly with integrated course 
design. It is our hope that by recognizing that these goals and their similarities and differ¬ 
ences are influenced by philosophical/conceptual stance, mixed research instructors and 
general research methodology instructors might reflect and re-evaluate ways in which 
philosophy, pedagogical strategies, and learning goals are reflected in course design. 
Therefore, one implication for mixed research course instructors might be to refer to Ear- 
ley^ (2007) recommendation of aligning learning goals with course objectives using a 
reflective practice that examines the learning process for ongoing course development. 
For example, instructors might reflect on Fink’s (2003) goals and assess their efforts in 
addressing student researcher identity - intentionally addressing the human dimension 
goal. 

Another implication emerging from our study is the idea that if students are to navigate 
successfully through a course in mixed research, instructors might initiate dialogue aimed 
at differentiating concepts from quantitative and qualitative research traditions in an ac¬ 
tive, integrative process specific to their fields of study. Further, instructors of research 
methodology courses in general (e.g., educational research) might reflect on ways to 
adopt mixed research concepts and philosophical diversity that might enhance creative 
thinking for students. As revealed by the findings for practical techniques in teaching 
mixed research (see Figures 4-7), many of the learning outcomes appear to overlap with 
concepts presented in general research methodology courses, such as quantitative sam¬ 
pling techniques, data analysis techniques, and ethical considerations. 

Due to the fact that our study aligned philosophical stance with teaching approaches, it 
was important for us to examine how our stated conceptual stance of dialectical pluralism 
impacted the interpretation of our findings. Dialectical pluralism is a stance whereby the 
researcher listens carefully and interprets the values, ideas, and concepts of multiple on¬ 
tologies, epistemologies, methodologies that include stakeholder and local perspectives 
(Johnson, McGowan, & Turner, 2011). Simply put, our dialectical pluralist stance guided 
the way that we situated the multiple perspectives of participants and focused on diversity 
in approach of teaching strategies in mixed research courses. 

In closing, the current study continues the dialogue called for by Creswell et al. (2003), 
Earley (2007), and Onwuegbuzie et al. (2011) to examine the rapidly developing field of 
mixed research for the next generation of researchers who are influenced by pedagogy 
and teaching approach. The U.S.-based leaders of mixed research who participated in our 
study delineated concepts, strategies, and course design elements specific to mixed re¬ 
search that they deemed important for teaching and learning mixed research. We encour¬ 
age future researchers to continue our line of inquiry so that instructors of mixed and 
general research methodology courses alike might better understand the impact of philo¬ 
sophical/conceptual stance, pedagogical strategies, and integration of learning goals for 
student development. 
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